--- title: Sensors keywords: fastai sidebar: home_sidebar summary: "Reads and packages IMU/GPS/Pressure-Humidity-Temperature sensor data over UART. Used for georectification. " description: "Reads and packages IMU/GPS/Pressure-Humidity-Temperature sensor data over UART. Used for georectification. " nb_path: "nbs/07_sensors.ipynb" ---
{% include tip.html content='This module can be imported using from openhsi.sensors import *' %}{% include warning.html content='Still experimental. Stay tuned. ' %}
The OpenHSI camera requires motion to generate 2D spatial datacubes. Yet, motion also introduces other artefacts that need georectification. To correct for spurious motion, we need to collect absolute orientation and geolocation of the camera simultaneously with the camera capture. This is where this module comes in. An IMU, GPS, and Pressure/Humidity/Temperature sensor needs to be read and recorded.
A Teensy 4.0 operates all the sensors, and devices using a Real Time Operating System (RTOS) like cooperative scheduler to run each component update at the desired frequency. By reducing the I/O, and CPU load on the main development board (the Raspberry Pi 4 with 8 GB RAM), the sensor updates are offloaded to a microcontroller with a real time clock to sync and timestamp each sensor measurement. The whole thing is assembled onto a PCB that stacks with the Raspberry Pi 4 and battery hat.
| Component | Rate (Hz) | Info |
|---|---|---|
| Teensy 4.0 (uC) | 24 MHz Clock | --- |
| NEO 9N (GPS) | 20 Hz | I2C @400 kHz (takes ~20 ms per update) |
| BNO055 (IMU) | 100 Hz | I2C_1 @400 kHz |
| BME280 (Air) | 100 Hz | I2C_1 @400 kHz |
| DS3231 (RTC) | 100 Hz | I2C_1 @400 kHz |
| XBee | 1 Hz | UART_1 @115,200 Hz (~2 ms per update) |
| Raspberry Pi 4 | packets @100 Hz | UART @921,600 Hz (~0.8 ms per update) |
| Start button | 4 Hz poll | button linked to LED notifying status |
An XBee is also programmed to check sensor status remotely during operation. This could be useful to diagnose any issues without being physically connected to the microcontroller. A basic streaming dashboard is included.
Each data packet contains timestamped sensor data. The item fields are then extracted from the raw binary serial stream. {% include note.html content='The Teensy runs a 32 bit Cortex-M7 so serial packets are padded. In other words, the data struct is padded in contiguous memory so a byte variable followed by float variable will include 3 unused bytes in-between so things are packed as 32 bits at a time. ' %}
The data packet is sent as a C struct so we need to decode the binary stream and interpret each byte as the corresponding C type. In each packet, there are status bytes to indicate which sensor has been updated.
It's fairly safe to assume that all data saving will occur on a separate storage device. I tested this with an SSD mounted over USB.
{% include tip.html content='The serial port for Raspberry Pi is "/dev/serial0". For the Jetson, it is "/dev/ttyTHS0".' %}{% include note.html content='GPS PPS callbacks are experimental. I haven’t found a way to use them effectively.' %}
Let's now test this using simulated ancillary sensor data packets.
We can simulate data packets for testing purposes. This will generate 77 data packets. You can then save the data - it will be cleaned up so each sensor has its own unique timestamp.
ss = SensorStream(baudrate = 921_600,
port = '/dev/serial0',
start_pin = 17,
ssd_dir = '.')
ss.packets = []
for i in tqdm(range(77)):
ss.packets.append(collect_sim(rtc_offset_ms=150))
time.sleep(0.01)
#ss.save()
We can also use an infinite loop to continuously save sensor data when a hardware button is latched. When data is saved, a summary plot of the data is also saved alongside. Here I specifically exclude the OpenHSI camera by not providing the argument cam_name to SensorStream.__init__.
#hide_output
ss = SensorStream(baudrate = 921_600,
port = '/dev/serial0',
start_pin = 17,
ssd_dir = '/media/pi/fastssd')
ss.master_loop()
Of course, you can also save ancillary sensor data with the OpenHSI camera datacubes - just provide cam_name and also the optional parameters in SensorStream.master_loop.
#hide_output
ss = SensorStream(baudrate = 921_600,
port = '/dev/serial0',
start_pin = 17,
ssd_dir = '/media/pi/fastssd',
cam_name="FlirCamera")
ss.master_loop(n_lines=256,
processing_lvl=2,
json_path="/media/pi/fastssd/cals/OpenHSI-FLIR01/OpenHSI-FLIR01_settings_Mono8_bin1.json",
pkl_path="/media/pi/fastssd/cals/OpenHSI-FLIR01/OpenHSI-FLIR01_calibration_Mono8_bin1.pkl",
preconfig_meta=None,
ssd_dir="/media/pi/fastssd",
switch_pin=17)
The ancillary sensor timestamps are different from the datacube along-track timestamps so some interpolation is needed. Here is a function that does that for you.
#hide_output
sd = SensorDashboard()
sd()
#hide_output
sd.run()
The SensorDashboard will save the data coming in which can be accessed in a pd.DataFrame. Here is some experimental data with noise added to the latitude/longitude points so the ESRI map loads.
sd.data_df.head(10)